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(54) Image processing apparatus and method for extracting feature of object 



(57) In this invention, a facial image is sensed from 
different directions (SO), pupil and nasal cavity regions 
are detected from the sensed facial image, feature 
points are detected from the detected regions (S1 - S4), 



a feature pattern is extracted on the basis of the feature 
points (S5), and the feature pattern is registered or is 
verified with a registered feature pattern to identify a per- 
son (S6-S12). 
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Description 

[0001] The present invention relates to an image processing apparatus and method, which are applied to entrance/ 
exit management of an important facility, or access management of a computer (terminal device) and are suitably 
5 applied to a personal authentication apparatus for authenticating a person on the basis of vital information (biometrics) 
such as a facial image. 

[0002] In recent years, in view of application to a human interface, security, and the like, a personal authentication 
technique based on vital information (biometrics) such as a facial image, or the like is becoming increasingly important, 
and various techniques have been developed. 
10 [0003] As biometrics, fingerprints, palmprints, voiceprints, and signature verifications, retina and iris scans, and the 
like are put into practical use. These schemes are categorized into "contact type" a person must directly touch a device, 
and "non-contact type" a person need not touch. 

[0004] Of the above examples of biometrics, fingerprints, palmprints, and signature verifications belong to "contact 
type". However, the category of signature verifications is slightly different from that of fingerprint verification and the 

15 like since a signature verification requires a person to take an action, i.e., to sign. 

[0005] Biometric that similarly requires person's action is voiceprint (speech) verification, but it belongs to "non- 
contact type". Retina and iris scans belong to "non-contact type" since a camera captures an image of a desired portion 
and the captured image is processed. However, since a retina is located at the bottom of an eyeball and must be 
scanned while the eye nearly contacts a lens, its category is close to "contact type". 

20 [0006] Note that the iris is present on the surface of the eyeball, and can be scanned by a camera at a separate 
position. However, since a micropattern must be scanned, the maximum distance between the eye and camera is 
naturally limited. 

[0007] On the other hand, facial image verification as non-contact biometric has received a lot of attention recently, 
and various techniques using this have been developed. A facial pattern has a larger scale than the aforementioned 
25 ins pattern. 

[0008] In a personal authentication apparatus that adopts facial image verification, one monitor camera is set at an 
appropriate position to capture a facial image of a full-face or nearly full-face pose, and the captured image undergoes 
pattern verification with facial image data registered in advance under similar conditions. 

[0009] Also, a system in which a camera is set at a relatively lower position to capture a looked-up facial image and 

30 to register and verify it has been proposed. 

[0010] For example, Jpn. Pat. Appln. KOKAI Publication No. 11-196398 discloses a technique that pertains to an 
image processing apparatus in which video cameras are laid out to make a given angle, a facial image of a full-face 
pose is stored in a full-face template memory, a facial image of a left-half-face pose is stored in a half-face template 
memory, and the correlation between images output from the video cameras and data stored in the memories is com- 

35 puted to determine the facial motion of a person (to be referred to as prior art 1 hereinafter). Furthermore, Fukui and 
Yamaguchi, "Facial Feature Point Extraction by Combining Shape Extraction and Pattern Verification", Journal of IEICE 
(D-ll), Vol. j-80-D-ll, No. 8, August 1997, proposes a method of extracting facial feature points of a pupil, nasal cavity, 
mouth edge, and the like from a moving image at high speed and high precision for facial recognition (to be referred 
to as prior art 2 hereinafter). 

40 [0011] Also, Yamaguchi, Fukui, and Maeda, "Facial Recognition System Using Moving Image", IEICE Transactions 
PRMU97-50, June 1997, proposes a personal identification method for facial recognition using a moving image (time- 
series images) in place of a single image (to be referred to as prior art 3 hereinafter). 

(1) An image processing apparatus of the present invention comprises a plurality of image sensing means for 
45 sensing object images from different directions, normalization means for extracting feature points from the object 

images sensed by the plurality of image sensing means, setting a feature region on the basis of the extracted 
feature points, segmenting the set feature region into a plurality of regions, computing an average value of bright- 
ness levels in each segmented region, and computing a feature pattern on the basis of the computed average 
value, registration means for registering the feature pattern computed by the normalization means as a feature 
50 pattern associated with a predetermined object, and verification means for specifying an object associated with 

the object image by comparing the feature pattern computed by the normalization means with the feature pattern 
registered in the registration means. 

(2) An image processing apparatus of the present invention comprises image input means for sensing an object 
image from different positions, and inputting a plurality of object images at different image sensing positions, feature 

55 extraction means for extracting feature patterns that represent features of an object from the plurality of object 

images input by the image input means, verification means for verifying the plurality of feature patterns extracted 
by the feature extraction means with a reference feature pattern which is registered in advance, and discrimination 
means for, when at least one of the plurality of feature patterns extracted by the feature extraction means matches 
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the reference feature pattern which is registered in advance as a result of verification of the verification means, 
determining that an object associated with that object image is a person himself or herself. 

(3) An image processing apparatus of the present invention comprises image input means for sensing an object 
image from different positions, and inputting a plurality of object images at different image sensing positions, input 

5 image determination means for determining an image sensing position of an object image to be used from the 

plurality of object images input by the image input means upon registration of a feature pattern, first feature ex- 
traction means for extracting a feature pattern which represents a feature of an object from the object image 
determined by the input image determination means, registration means for registering the feature pattern extracted 
by the first feature extraction means as a reference feature pattern associated with the object in correspondence 

10 with position information indicating the image sensing position of the corresponding object image, verification image 

selection means for selecting an object image at an image sensing position, which corresponds to the position 
information registered together with the feature pattern of the object to be verified registered in the registration 
means, of the plurality of object images input by the image input means upon verification of a feature pattern, 
second feature extraction means for extracting a feature pattern which represents a feature of the object from the 

15 object image selected by the verification image selection means, and verification means for specifying an object 

associated with the object image by verifying the feature pattern extracted by the second feature extraction means 
with the feature pattern of the object to be verified registered in the registration means. 

(4) An image processing method of the present invention comprises the first step of sensing object images from 
different directions, the second step of extracting feature points from the object images sensed in the first step, 

20 setting a feature region on the basis of the extracted feature points, segmenting the set feature region into a plurality 

of regions, computing an average value of brightness levels in each segmented region, and computing a feature 
pattern on the basis of the computed average value, the third step of registering the feature pattern computed in 
the second step as a feature pattern associated with a predetermined object, and the fourth step of specifying an 
object associated with the object image by comparing the feature pattern computed in the second step with the 

25 feature pattern registered in the third step. 

(5) An image processing method of the present invention comprises the first step of sensing an object image from 
different positions, and inputting a plurality of object images at different image sensing positions, the second step 
of extracting feature patterns that represent features of an object from the plurality of object images input in the 
first step, the third step of verifying the plurality of feature patterns extracted in the second step with a reference 

30 feature pattern which is registered in advance, and the fourth step of determining, when at least one of the plurality 

of feature patterns extracted in the second step matches the reference feature pattern which is registered in ad- 
vance as a result of verification of the third step, that an object associated with that object image is a person himself 
or herself. 

(6) An image processing method of the present invention comprises the first step of sensing an object image from 
35 different positions, and inputting a plurality of object images at different image sensing positions, the second step 

of determining an image sensing position of an object image to be used from the plurality of object images input 
in the first step upon registration of a feature pattern, the third step of extracting a feature pattern which represents 
a feature of an object from the object image determined in the second step, the fourth step of registering the feature 
pattern extracted in the third step as a reference feature pattern associated with the object in correspondence with 

40 position information indicating the image sensing position of the corresponding object image, the fifth step of se- 

lecting an object image at an image sensing position, which corresponds to the position information registered 
together with the feature pattern of the object to be verified registered in the fourth step, of the plurality of object 
images input in the first step upon verification of a feature pattern, the sixth step of extracting a feature pattern 
which represents a feature of the object from the object image selected in the fifth step, and the seventh step of 

45 specifying an object associated with the object image by verifying the feature pattern extracted in the sixth step 

with the feature pattern of the object to be verified registered in the fourth step. 

[001 2] This summary of the invention does not necessarily describe all necessary features so that the invention may 
also be a sub-combination of these described features. 
50 [001 3] The invention can be more fully understood from the following detailed description when taken in conjunction 
with the accompanying drawings, in which: 

FIG. 1 is a schematic block diagram showing the arrangement of an image processing apparatus according to the 
first embodiment of the present invention; 
55 FIG. 2 shows an example of a camera layout in the first embodiment; 

FIG. 3 shows examples of facial images captured by respective cameras in the first embodiment; 

FIG. 4 is a flow chart for explaining facial image registration and verification processes by a processor according 

to the first embodiment; 



3 




EP1 134 691 A2 



FIG. 5 shows examples of feature point extraction results in the first embodiment; 
FIG. 6 is a view for explaining a normalization process in the first embodiment; 
FIG. 7 is a view for explaining a normalization process in the first embodiment; 

FIG. 8 is a view for explaining a scheme for computing a feature pattern in a vector format in the first embodiment; 
5 FIG. 9 shows an example of a camera layout in an image processing apparatus according to the second embod- 

iment of the present invention; 

FIG. 10 shows an example of extracted facial images and feature points in the second embodiment; 

FIG. 11 is a view for explaining a normalization process when three feature points are obtained in the second 

embodiment; 

10 FIG. 12 is a view for explaining a normalization process when three feature points are obtained in the second 

embodiment; 

FIG. 1 3 shows an example of a camera layout in an image processing apparatus according to the third embodiment 
of the present invention; 

FIG. 14 shows an example of a camera layout in an image processing apparatus according to the sixth embodiment 
15 of the present invention; 

FIG. 15 is a chart for explaining a registration process according to the seventh embodiment of the present inven- 
tion; 

FIG. 16 is a chart for explaining a verification process according to the seventh embodiment; 
FIG. 17 shows an example of a camera layout in an image processing apparatus according to the eighth embod- 
20 iment of the present invention; 

FIG. 18 is a schematic block diagram showing the arrangement of the image processing apparatus according to 
the eighth embodiment; and 

FIG. 19 is a flow chart for explaining a verification computation process by a processor according to the eighth 
embodiment. 

25 

[0014] Preferred embodiments of the present invention will be described hereinafter with reference to the accompa- 
nying drawings. 

[0015] The first embodiment of the present invention will be described below. 

[0016] FIG. 1 shows the arrangement of an image processing apparatus according to the first embodiment. Referring 
30 to FIG. 1, a plurality of video cameras (to be simply referred to as cameras hereinafter) 1-1, 1-2,..., 1-N as image 
sensing means for sensing an object image line up vertically to have a given spacing. This embodiment uses video 
cameras that can sense monochrome images. The outputs from the cameras 1-1, 1-2,..., 1-N are respectively con- 
nected to capture boards 2-1, 2-2 2-N. 

[0017] That is, a video signal (analog data) 1 from the camera 1-1 is converted into digital data by an A/D converter 
35 2-la, and the digital data is temporarily stored in an image memory 2-1 b. The capture boards 2-2,..., 2-N similarly 
comprise A/D converters and image memories (not shown for the sake of simplicity), and video signals 2 to N undergo 
similar processes in the capture boards 2-2,..., 2-N. 

[0018] When each of the cameras 1-1, 1-2,..., 1-N comprises a USB (Universal Serial Bus) interface, a USB interface 
is added in place of the A/D converter 2-la. The same applies to a case wherein the camera comprises a digital I/O 

40 interface such as IEEE1394 other than USB. 

[0019] The capture boards 2-1 , 2-2 2-N, a processor 4, a work memory 5, a display board 6, and a dictionary 7 

are connected to each other via a system bus 3 so as to be able to communicate with each other. Furthermore, a 
display 8 is connected to the display board 6. The dictionary 7 is a registration means in which a plurality of dictionary 
data (reference feature patterns) are registered (stored). The display 8 outputs, e.g., a verification result. 

45 [0020] The processor 4 sends a control signal for taking synchronization to the cameras 1-1, 1-2,..., 1-N, receives 
digital data that pertain to facial images sent from the cameras 1-1, 1-2,..., 1-N, and executes facial image registration, 
verification, and determination processes (to be described later) of the received data using the work memory 5 and 
dictionary 7. 

[0021] Note that processors exclusively used to process images sensed by the cameras 1-1, 1-2 1-N may be 

50 parallelly arranged to achieve high-speed processing. 

[0022] FIG. 2 shows details of an example of the layout of the cameras 1-1, 1-2,..., 1-N, and FIG. 3 show examples 
of facial images sensed by the cameras 1-1, 1-2,..., 1-N. 

[0023] In this example, three cameras 1-1, 1-2, and 1-3 line up vertically to have an appropriate spacing, and an 
object (face) F is located in front of these cameras 1-1, 1-2, and 1-3. With this layout, the cameras 1-1, 1-2, and 1-3 
55 respectively sense and capture a looked-down facial image (see (a) of FIG. 3), a frontal facial image (see (b) of FIG. 
3), and a looked-up facial image (see (c) of FIG. 3). 

[0024] Note that the cameras 1-1,1 -2, and 1 -3 are illustrated as independent input devices. Alternatively, the cameras 
1-1,1 -2, and 1 -3 may be stored in a single housing, so that they appear as a single input device for the user. 
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[0025] The facial image registration and verification processes by the processor 4 of the image processing apparatus 
of the first embodiment will be described in detail below with reference to the flow chart shown in FIG. 4. 
[0026] Note that the flow chart in FIG. 4 shows processes to be individually done for facial images captured by the 
cameras 1-1, 1-2, and 1-3. 

5 [0027] The cameras 1-1,1 -2, and 1 -3 sense an object image from different directions (SO). The processor 4 seeks 
a facial image region from the entire input image (S1). Pupil and nasal cavity regions, which are regarded as substan- 
tially circular regions, are detected, and the central positions of these regions are detected as feature points of the 
facial image (S2). Note that the processes in steps S1 and S2 can adopt the conventional method described in, e.g., 
prior art 2. The detection results of the feature points are as shown in (a), (b), and (c) of FIG. 5, and "x" marks in these 

10 figures indicate the detected feature points. Note that (a) of FIG. 5 corresponds to the facial image captured by the 
camera 1-1, (b) of FIG. 5 corresponds to the facial image captured by the camera 1-2, and (c) of FIG. 5 corresponds 
to the facial image captured by the camera 1-3. 

[0028] Then, a normalization process including extraction of a feature region (S3) and segmentation of the region 

(54) is executed. The normalization process in steps S3 and S4 is a core of the facial image registration and verification 
is processes in this embodiment. The normalization process will be described in detail below with reference to FIGS. 6 

and 7. 

[0029] As shown in FIGS. 6 and 7, the normalization process takes different procedures depending on the number 
of obtained feature points (two or four points). If the number of obtained feature points is other than 2 or 4, the normal- 
ization process is skipped. 

20 [0030] A feature region is defined as a rectangle obtained by enlarging a rectangle specified by a plurality of feature 
points at a magnification set for each camera. More specifically, when two feature points are obtained, as shown in, 
e.g., (a) of FIG. 6, a rectangular region (p1, p2, p3, p4) obtained by further enlarging a rectangle formed by giving a 
predetermined width to a line segment flf2 at a predetermined magnification is defined as a feature region, as shown 
in (b) of FIG. 6. In this example, the obtained feature region is broken up into 10x5 small rectangular regions, as 

25 shown in (c) of FIG. 6. 

[0031] On the other hand, when four feature points are obtained, as shown in, e.g., (a) of FIG. 7, a rectangular region 
(q1 , q2, q3, q4) obtained by further enlarging a rectangle formed by giving a predetermined width to line segments f1 f2 
and f3f4 at a predetermined magnification is defined as a feature region, as shown in (b) of FIG. 7. In this example, 
the obtained feature region is broken up into 15x15 small rectangular regions, as shown in (c) of FIG. 7. 

30 [0032] In this embodiment, a rectangular feature region is used. However, the present invention is not limited to this 
as long as a region can be uniquely computed from two or four feature points. On the other hand, the feature region 
is broken up by equally segmenting the respective sides of the rectangle. Alternatively, the segmented small areas 
may have variable sizes, i.e., areas, in consideration of feature strength. The number of equally segmented areas is 
not limited to the aforementioned values. As a characteristic feature of the image processing apparatus of this embod- 

35 iment, parameters of the normalization process can be optimized in units of cameras. 

[0033] The description will revert to FIG. 4. A feature pattern for registration and verification is computed (extracted) 

(55) . In this process, the average values of brightness levels are computed in units of partial regions broken up in step 
S4, and are arranged in an order (raster scan order) shown in FIG. 8 to express facial feature pattern data (to be simply 
referred to as a feature vector hereinafter): 

40 (V11, V12 Vnm-1, Vnm) 

[0034] As a modification of the process in step S5, pixels in the feature region undergo density correction prior to 
the average value computation to normalize the density range. Alternatively, a feature emphasis filter process such as 
a differential process may be executed. Furthermore, other density or graphic feature amounts such as most frequent 
values (mode values), principal directions of edges (extracted by the differential process), and the like may be used in 

45 place of the average values as feature amounts in segmented small areas. 

[0035] It is then checked if registration or verification is made (S6). If registration is to be made, a plurality of feature 
patterns are stored (S7, S8), dictionary data are generated using the scheme (partial space method) of prior art 2 (S9), 
and a procedure for registering dictionary data in the dictionary 7 together with personal attribute information is done 
(S10), thus ending registration. 

50 [0036] On the other hand, if verification is to be made, a verification computation process using the scheme (partial 
space method) of prior art 2 is done (S11 ), and a verification level output process is then executed (S12). 
[0037] As described above, according to the first embodiment, a single input facial image (still image) is to be proc- 
essed. In consideration of variations of the input conditions, a plurality of successive facial images (moving image) 
may be captured at a given time interval, and may be buffered on the image memory 2-lb of the capture board 2-1 

55 shown in FIG. 1 . The method of making verification computation using a moving image (time-series images) is described 
in detail in prior art 3 (mutual partial space method). 

[0038] In the flow upon processing a moving image, the processes in steps S1 to S4 can be the same as those in 
the flow chart in FIG. 4, and the contents of only steps S9 and S1 0 need only be changed. That is, after a predetermined 
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number of feature patterns are stored as in registration, a statistical process such as main component analysis or the 
like is done based on these plurality of feature vectors, and a pattern verification process is then executed. 
[0039] A characteristic feature of the first embodiment mentioned above lies in integrated control of processes of 
facial images from the cameras 1-1, 1-2, and 1-3. To summarize, the processor 4 executes the following control. 

5 [0040] First, the cameras 1-1 and 1-2 detect the central positions of the eyes as two (right and left) feature points, 
and the camera 1-3 detects the two central points of the nasal cavities, thus obtaining a total of four feature points. 
[0041] If this condition is not satisfied, and at least one of these cameras fails to detect an image, it is determined 
that the process at that time is not successful, and the process is interrupted immediately. Then, the process is retried 
or an error message is displayed for the user. 

10 [0042] Second, upon registration, feature vectors corresponding to facial images from all the cameras 1-1 , 1-2, and 
1-3 are registered as a set of person data. 

[0043] Third, if verification levels between facial images captured by all the cameras 1-1 , 1-2, and 1-3 and registered 
data of a given person are output as M1, M2, and M3, it is determined that "that person is identified" only when a 
condition: 

15 M1 >= K1 , and M2 >= K2, and M3 >= K3 

where K1, K2, and K3 are predetermined threshold values is satisfied. 

[0044] Upon searching for a person corresponding to the input facial image, of persons whose data satisfy the above 

condition (if no person data satisfies the above condition, "no match" is determined), a person corresponding to a 

maximum average value of verification levels, i.e., 
20 (M1 +M2 + M3)/3 

is finally determined to be a match. 

[0045] The second embodiment of the present invention will be described below. 

[0046] The basic arrangement of an image processing apparatus according to the second embodiment is substan- 
tially the same as that in FIG. 1 (first embodiment), except that the cameras 1-1, 1-2, and 1-3 line up horizontally. 
25 [0047] The layout of these cameras 1-1,1 -2, and 1-3 is as shown in FIG. 9. That is, as shown in FIG. 9, the cameras 
1-1, 1-2, and 1-3 line up horizontally at an appropriate spacing so as to be angled slightly upward. With this layout, the 
cameras 1-1, 1-2, and 1-3 respectively sense and capture a facial image of a left half-face pose, a looked-up facial 
image of a full-face pose, and a facial image of a right half-face pose. 

[0048] Examples of facial images input from the cameras 1-1 , 1-2, and 1-3 and feature points extracted therefrom 
30 are as shown in (a), (b), and (c) of FIG. 10, and "x" marks in these figures indicate the detected feature points. Note 
that (a) of FIG. 10 corresponds to the camera 1-1, (b) of FIG. 10 corresponds to the camera 1-2, and (c) of FIG. 10 
corresponds to the camera 1-3. 

[0049] The process executed by the processor 4 is substantially the same as that in the first embodiment, except 
that the process executed upon obtaining two feature points in the first embodiment is replaced by that executed upon 
35 obtaining three feature points, as will be described in detail below. 

[0050] Examples of the normalization process upon obtaining three feature points will be explained below with ref- 
erence to FIGS. 11 and 12. 

[0051] More specifically, when three feature points are obtained from the right half-face facial image, as shown in 

(a) of FIG. 11, a rectangular region (p1, p2, p3, p4) based on a triangle (f1, f2, f3) is defined to be a feature region, as 
40 shown in (b) of FIG. 11 . In this example, this feature region is broken up into 10x15 small rectangular regions, as 

shown in (c) of FIG. 11. 

[0052] Likewise, when three feature points are obtained from the left half-face facial image, as shown in (a) of FIG. 
12, a rectangular region (q1, q2, q3, q4) based on a triangle (f1, f2, f3) is defined to be a feature region, as shown in 

(b) of FIG. 12. In this example, this feature region is broken up into 10x15 small rectangular regions, as shown in (c) 
45 of FIG. 12. 

[0053] In order to obtain a rectangular region from three feature points, a line which is parallel to a line segment f1f2 
and passes through f3 is drawn, two points separated a given width from f3 are set on the line on the two sides of f3, 
a rectangle having as vertices a total of four points including these two points, and f1 and f2 can be enlarged at a 
predetermined magnification. At this time, the magnification is set in advance for each camera. However, the present 
50 invention is not limited to this method, and a feature region need only be defined in advance so that it can be uniquely 
determined. 

[0054] The third embodiment of the present invention will be described below. 

[0055] The basic arrangement of an image processing apparatus according to the third embodiment is substantially 
the same as that in FIG. 1 (first embodiment), except that nine cameras 1-1, 1-2,..., 1-9 are set two-dimensional ly. 

55 [0056] The layout of these cameras 1-1 , 1-2 1-9 is as shown in FIG. 13. That is, as shown in FIG. 13, the cameras 

1-1, 1-2, and 1-3 respectively capture slightly looked-down facial images of a left half-face pose, full-face pose, and 
right half-face pose. The cameras 1-4, 1-5, and 1-6 respectively capture facial images of a left half-face pose, full-face 
pose, and right half-face pose. Furthermore, the cameras 1-7, 1-8, and 1-9 respectively capture slightly looked-up 
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facial images of a left half-face pose, full-face pose, and right half-face pose. 

[0057] Note that the process executed by the processor 4 according to the third embodiment can be implemented 
by combining the processes of the first and second embodiments described above. 
[0058] The fourth embodiment of the present invention will be described below. 
5 [0059] The basic arrangement of an image processing apparatus according to the fourth embodiment is substantially 
the same as that in FIG. 1 (first embodiment). Also, the camera layout is the same as that in one of the first to third 
embodiments. 

[0060] The process executed by the processor 4 according to the fourth embodiment can be realized by adding a 
feature vector extraction process to that explained in the first to third embodiments. That is, an integration process of 
10 feature vectors extracted from a plurality of facial images is added. 

[0061] More specifically, when a plurality of cameras 1-1 , 1-2 1-n are deployed, if feature vectors: camera 

1-1: (V11 V1L1)(L1 dimension) 

camera 1-2: (V21 V2L2) (L2 dimension) 

15 camera 1-n: (Vn1 VnLn) (Ln dimension) 

are computed from the outputs of the cameras 1-1, 1-2,..., 1-n, these feature vectors are integrated by coupling in turn: 

(V11 V1L1, V21,.., V2L2 Vn1 VnLn) 

(L1+L2+...+Ln) dimension 

[0062] As a result of such feature vector integration, the same registration and verification processes (partial space 
20 .method) as in a single camera described in, e.g., prior art 2, can be made although a plurality of cameras are used. 
[0063] The fifth embodiment of the present invention will be described below. 

[0064] The basic arrangement of an image processing apparatus according to the fifth embodiment is substantially 
the same as that in FIG. 1 (first embodiment). Also, the camera layout is the same as that in one of the first to third 
embodiments. 

25 [0065] The process executed by the processor 4 according to the fifth embodiment can be realized by adding a 
feature vector extraction process to that explained in the first to third embodiments. Note that input facial images are 
time-serially captured at given time intervals, and a process to be added for the purpose of improving the verification 
precision is an integration process of feature vectors on the time series, as will be described below. 
[0066] More specifically, when feature vectors at times 1, 2,..., s (assume that feature patterns from a plurality of 
30 cameras have already been integrated by the method of the fourth embodiment, and L represents that total dimension) 
are computed, the input feature vectors 

time1:(V11 V1L) 

time 2: (V21 V2L) 

35 times: (Vs1,..., VsL) 

are integrated on the time series to obtain an average vector described by: 
(vT, v2, - ,vL) 



vi = Y vt - i (EQ1) 

3 t =1 

when a moving image is used. 

45 [0067] As a result of integration on the time series, the method of prior art 2 can be used as in case of a single 
camera. The method of prior art 3 may be applied without executing such process for obtaining the average feature 
vector. 

[0068] As described above, according to the first to fifth embodiments, a plurality of cameras are set within a range 
in which they satisfy a given condition so as to simultaneously capture facial images of an identical person, and a 
50 correction process is done on the basis of a total of four pieces of feature point position information of pupils and nasal 
cavities, thus improving the verification precision while maintaining the load on the user light as in the prior art. 
[0069] When feature patterns extracted from a plurality of facial images are integrated to one to execute the same 
pattern verification process as in the prior art, increases in complicated computation and processing cost can be min- 
imized. 

55 [0070] The sixth embodiment of the present invention will be described below. 

[0071] The basic arrangement of an image processing apparatus according to the sixth embodiment is substantially 

the same as that in FIG. 1 (first embodiment), except for the layout and use method of cameras 1-1, 1-2 1-N, and 

a corresponding integration process. 
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[0072] FIG. 14 shows an example wherein two cameras 1-1 and 1-2 are laid out vertically. As shown in FIG. 14, the 
cameras 1-1 and 1-2 line up vertically at an appropriate spacing to be angled slightly upward. An object F is located 
in front of these cameras 1-1 and 1-2. With this layout, the cameras 1-1 and 1-2 sense slightly looked-up facial images 
of a full-face pose. In FIG. 14, reference numeral 9-1 denotes an illumination lamp arranged above the camera 1-1; 

5 and 9-2, an illumination lamp arranged below the camera 1-2. 

[0073] In the example shown in FIG. 14, the camera 1-1 is set at a level that assumes the average height of an adult, 
and the camera 1-2 is set at a level that assumes the height of an adult equal to or lower than the average height or 
a child. Even when a very tall person falls outside the field of view of the camera, his or her facial image can be input 
when he or she bends down. An infant may be excluded from the process, or his or her facial image may be input 

10 using a stool or while being lifted in his or her parent's arms. Also, the camera 1-2 can be used for a handicapped 
person in a wheelchair. 

[0074] The integration process is executed by the processor 4 according to the sixth embodiment as follows. That 
is, in the first embodiment, when feature point detection based on at least one of facial images from a plurality of 
cameras fails, it is determined that the process at that time is not successful, and the process is interrupted immediately. 
15 Then, the process is retried or an error message is displayed for the user. In the sixth embodiment, verification is done 
for a facial image from which feature points can be successfully detected, and if verification of at least one facial image 
is successful, it is determined that "the person is identified". 

[0075] More specifically, if verification levels between facial images captured by the cameras 1-1 and 1-2, and reg- 
istered data (dictionary data) of a given person are output as M1 and M2, it is determined that "that person is identified" 
20 when a condition: 

M1 >= K1 or M2 >= K2 

where K1 and K2 are predetermined threshold values is satisfied. 
[0076] With this process, flexible verification can be done in correspondence with individual height differences. 
[0077] In the sixth embodiment, two cameras are used. However, three or more cameras may be used. In addition, 
25 only when verification of a predetermined number or more of facial images is successful in place of that of one of facial 
images, it is determined that "the person is identified". That is, if verification levels between facial images captured by 

the cameras 1-1 , 1-2 1-N and registered data (dictionary data) of a given person are output as M1 , M2 Mn, it is 

determined that "that person is identified" when a condition: 

R >= T (T > 2) 

30 where R is the number of times Mi >+ Ki (i = 1, 2,..., n) holds, and Ki and T are predetermined threshold values 

is satisfied. 

[0078] The seventh embodiment of the present invention will be described below. 

[0079] In the seventh embodiment, a camera used to obtain a facial image is determined in the registration and 
verification processes by the processor 4 in the sixth embodiment. That is, upon registration, a camera used to obtain 
35 a facial image is determined, and a facial image is registered together with the camera number. More specifically, as 
shown in FIG. 15, a registrant determines one of the cameras 1-1 and 1-2 which he or she would like to use depending 
on his or her physical construction and height, and selects that camera (S41). Also, the registrant inputs an ID code 
via a keyboard, card, or the like as his or her own identification information. 

[0080] A facial image input by the selected camera undergoes the aforementioned image process to extract a feature 
40 pattern (S1 to S5), and the extracted feature pattern is registered in the dictionary 7 together with the input ID code of 
the registrant, and the camera number as the position information of the selected camera (i.e., position information 
indicating the image sensing position of an object image) (S7 to S10). 

[0081] Note that the camera to be used may be determined by the registrant himself or herself, but a camera that 
can obtain a facial image from which feature points corresponding to two pupils and two nasal cavities are extracted, 
45 and the central portion of which is closest to these positions may be automatically determined. In this example, infor- 
mation indicating a camera to be used (camera number as camera position information) can also be used as personal 
information, thus setting higher security level. 

[0082] Upon verification, a verification process is done for only a facial image obtained from a camera corresponding 
to the registered camera number. More specifically, as shown in, e.g., FIG. 16, a person to be verified inputs his or her 
so own ID code via a keyboard, card, or the like, and the dictionary 7 is searched for the camera number and feature 
pattern registered in correspondence with that ID code (S42). 

[0083] A camera corresponding to the camera number retrieved from the dictionary 7 is selected (S43), and a facial 
image input by the selected camera undergoes the aforementioned image process to extract a feature pattern (S1 to 
S5). The extracted feature pattern is verified with the feature pattern retrieved from the dictionary 7 to determine if the 
55 person to be verified is the person himself or herself (S11 , S12). 

[0084] In the seventh embodiment, two cameras are used. Alternatively, three or more cameras may be used. In 
addition, only one camera is selected, but when three or more cameras are used, two or more cameras may be selected. 
[0085] The eighth embodiment of the present invention will be described below. 
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[0086] In the first to seventh embodiments, a required number of cameras are set in advance. However, in the eighth 
embodiment, a camera moving mechanism is arranged, and moves a single camera to a predetermined position so 
as to execute the aforementioned process. In this case, an image capture time is slightly prolonged, but only one 
camera is required, resulting in simple setting. 

5 [0087] FIG. 17 shows an example wherein one camera 1 is set. More specifically, as shown in FIG. 17, the camera 
1 is fixed to a vertical guide rail 10 to be slidable vertically, and is moved by a camera moving mechanism (not shown) 
to a first position P1 (corresponding to the position of the camera 1-1 in FIG. 14) or a second position P2 (corresponding 
to the position of the camera 1-2 in FIG. 14). The camera 1 is set to be angled slightly upward. 
[0088] The basic arrangement of an image processing apparatus according to the eighth embodiment is substantially 

10 the same as that in FIG. 1 (first embodiment), except that the number of sets of cameras and capture boards is reduced 
to one, and a camera position controller 12 for controlling a camera moving mechanism 11 that moves the camera 1, 
and a communication board 13 for connecting the camera position controller 12 to the system bus 3 are added, as 
shown in FIG. 18. 

[0089] The process executed by the processor 4 according to the eighth embodiment is basically the same as that 
15 in FIG. 4 (first embodiment), except for the contents of the verification computation process in step S 1 1 . The verification 
computation process in step 11 will be described in detail below with reference to the flow chart shown in FIG. 19. 
[0090] The operation varies depending on whether 1 : 1 or 1 : N verification is done (S21 ). Note that 1 : 1 verification 
is a verification process executed when a person to be verified inputs his or her own ID code to designate registered 
data to be verified, and 1 : N verification is a verification process executed when a person to be verified does not input 
20 any ID code to select all registered data as data to be verified. 

[0091] If 1 : 1 verification is made, the ID code input by the person to be verified is read (S22). Registered data (partial 
space) corresponding to the read ID code is read from the dictionary 7 (S23). A verification level between the spatial 
space of the registered data and an input partial space is computed to verify using, e.g., the partial space method 
(S24). The verification level is compared with a predetermined threshold value (S25, S26) to output a 1 : 1 verification 
25 result (S27). 

[0092] If 1 : N verification is made, all registered data to be verified are read from the dictionary 7 (S28). Verification 
levels with these registered data are computed (S29). A maximum one of the computed verification levels is selected 
(S30) and is output as a 1 : N verification result (S32). 

[0093] In this case, if the maximum verification level may undergo a threshold value process like in step S31 bounded 
30 by the broken line in FIG. 1 9, it can be checked if the verification result is correct (S33). For example, if the verification 
level is too low, it is determined that no match is found. 

[0094] Note that 1 : N verification is a verification process executed when a person to be verified does not input any 

ID code to select all registered data as data to be verified, and corresponds to the verification process in the first to 

sixth embodiments described above. 
35 [0095] In the eighth embodiment, only one camera is used, but two or more cameras may be used. Furthermore, 

only one camera is moved. However, when two or more cameras are used, these cameras may be moved. 

[0096] The embodiments of the present invention have been explained, but the present invention is not limited to 

these specific embodiments and various changes may be made without departing from the scope of the invention. For 

example, the aforementioned embodiments may be appropriately combined. 
40 [0097] In the above embodiments, a camera of a normal visible wavelength range is used. Alternatively, an infrared 

ray camera of an infrared range may be used, or a rangefinder that can obtain distance information (depth information) 

may be used. Also, image input devices of different types may be mixed. 

[0098] As described above, according to the present invention, an image processing apparatus and method, which 
can reduce the load on the user upon registration and verification, can greatly improve verification performance, and 
45 can minimize increases in complicated computation and processing cost without largely changing a pattern verification 
process algorithm from a single direction can be provided. 

[0099] By contrast, the technique of prior art 1 independently processes images although it uses a plurality of cam- 
eras, and a function is limited to detection of the face direction. In the techniques of prior arts 2 and 3, since the 
countenance of a person has three-dimensional shape information, two-dimensional facial image information captured 
50 from only one direction alone limits personal identification performance, and practical verification performance cannot 
become so high compared to fingerprint verification, retina and iris scans, and the like. 



Claims 

55 

1. An image processing apparatus characterized by comprising: 

image processing means (1-1, 1-2,..., 1-N, 2-1, 2-2,..., 2-N, 4) for sensing object images from different di- 
rections, extracting feature points from the sensed object images, and computing a feature pattern on the basis 
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of the extracted feature points. 

An apparatus according to claim 1 , characterized by comprising: 

a plurality of image sensing means (1-1, 1-2 1-N) for sensing object images from different directions; and 

normalization means (4) for extracting feature points from the object images sensed by said plurality of image 
sensing means, setting a feature region on the basis of the extracted feature points, segmenting the set feature 
region into a plurality of regions, computing predetermined information in each segmented region, and com- 
puting a feature pattern on the basis of the computed predetermined information. 

An apparatus according to claim 1 , characterized by comprising: 

a plurality of image sensing means (1-1, 1-2 1-N) for sensing object images from different directions; 

normalization means (4) for extracting feature points from the object images sensed by said plurality of image 
sensing means, setting a feature region on the basis of the extracted feature points, segmenting the set feature 
region into a plurality of regions, computing an average value of brightness levels in each segmented region, 
and computing a feature pattern on the basis of the computed average value; 

registration means (4, 7) for registering the feature pattern computed by said normalization means as a feature 
pattern associated with a predetermined object; and 

verification means (4, 7) for specifying an object associated with the object image by comparing the feature 
pattern computed by said normalization means with the feature pattern registered in said registration means. 

An apparatus according to claim 3, characterized in that said plurality of image sensing means line up vertically, 
and 

said normalization means computes the feature pattern using one of a feature point group including central 
points of right and left pupils and central points of right and left nasal cavities of the object image, and a feature 
point group including central points of right and left pupils of the object image. 

An apparatus according to claim 3, characterized in that said plurality of image sensing means line up horizontally, 
and 

said normalization means computes the feature pattern using one of a feature point group including central 
points of right and left pupils and central points of right and left nasal cavities of the object image, a feature point 
group including central points of right and left pupils and a central point of a left nasal cavity of the object image, 
and a feature point group including central points of right and left pupils and a central point of a right nasal cavity 
of the object image. 

An apparatus according to claim 3, characterized in that said plurality of image sensing means line up vertically 
and horizontally, and 

said normalization means computes the feature pattern using one of a feature point group including central 
points of right and left pupils and central points of right and left nasal cavities of the object image, a feature point 
group including central points of right and left pupils and a central point of a left nasal cavity of the object image, 
a feature point group including central points of right and left pupils and a central point of a right nasal cavity of 
the object image, and a feature point group including central points of right and left pupils of the object image. 

An apparatus according to claim 3, characterized in that said normalization means extracts feature vectors of 
different dimensions from respective object images sensed by said plurality of image sensing means, and arranges 
the extracted feature vectors of different dimensions in turn to integrate them as a multi-dimensional feature pattern. 

An apparatus according to claim 3, characterized in that said normalization means captures object images sensed 
by said plurality of image sensing means at predetermined time intervals, computes feature patterns of the object 
images of identical times, and arranges feature patterns of different times in turn to integrate them as a time-series 
feature pattern. 

An apparatus according to claim 1 , characterized by comprising: 

image input means (1-1, 1-2,..., 1-N) for sensing an object image from different positions, and inputting a 
plurality of object images at different image sensing positions; 

feature extraction means (4) for extracting feature patterns that represent features of an object from the plurality 
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of object images input by said image input means; 

verification means (4, 7) for verifying the plurality of feature patterns extracted by said feature extraction means 
with a reference feature pattern which is registered in advance; and 

discrimination means (4) for, when at least one of the plurality of feature patterns extracted by said feature 
5 extraction means matches the reference feature pattern which is registered in advance as a result of verification 

of said verification means, determining that an object associated with that object image is a person himself or 
herself. 

10. An apparatus according to claim 9, characterized in that said image input means has a plurality of image sensing 
10 means which are set in advance at a plurality of predetermined positions, and sense an object image from a plurality 

of different positions, and inputs a plurality of object images at different image sensing positions using said plurality 
of image sensing means. 

An apparatus according to claim 9, characterized in that said feature extraction means comprises: 

feature point detection means for detecting feature points of an object from the input object image; 
feature region setting means for setting a feature region on the basis of the feature points detected by said 
feature point detection means; 

region segmentation means for segmenting the feature region set by said feature region setting means into 
a plurality of regions; and 

feature pattern extraction means for computing brightness average values in the regions segmented by said 
region segmentation means, and extracting a feature pattern which represents a feature of the object on the 
basis of the brightness average values. 

25 12. An apparatus according to claim 1 , 
characterized by comprising: 

image input means (1-1, 1-2,..., 1-N) for sensing an object image from different positions, and inputting a 
plurality of object images at different image sensing positions; 
30 input image determination means (4) for determining an image sensing position of an object image to be used 

from the plurality of object images input by said image input means upon registration of a feature pattern; 
first feature extraction means (4) for extracting a feature pattern which represents a feature of an object from 
the object image determined by said input image determination means; 

registration means (4, 7) for registering the feature pattern extracted by said first feature extraction means as 
35 a reference feature pattern associated with the object in correspondence with position information indicating 

the image sensing position of the corresponding object image; 

verification image selection means (4) for selecting an object image at an image sensing position, which cor- 
responds to the position information registered together with the feature pattern of the object to be verified 
registered in said registration means, of the plurality of object images input by said image input means upon 
4 o verification of a feature pattern; 

second feature extraction means (4) for extracting a feature pattern which represents a feature of the object 
from the object image selected by said verification image selection means; and 

verification means (4, 8) for specifying an object associated with the object image by verifying the feature 
pattern extracted by said second feature extraction means with the feature pattern of the object to be verified 
45 registered in said registration means. 

13. An apparatus according to claim 1, characterized in that an object image is input at different image sensing 
positions by moving at least one image sensing means to a plurality of predetermined positions and sensing an 
object image at respective positions. 

50 

14. An apparatus according to claim 1 , 
characterized by comprising: 

a plurality of image sensing means, respectively set in advance at a plurality of predetermined positions, for 
55 sensing an object image from a plurality of different positions; 

determination means for determining a position of the image sensing means to be used of said plurality of 
image sensing means upon registration of a feature pattern; 

first feature extraction means for extracting a feature pattern which represents a feature of an object from the 
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object image obtained by the image sensing means determined by said determination means; 
registration means for registering the feature pattern extracted by said first feature extraction means as a 
reference feature pattern associated with the object in correspondence with position information indicating the 
position of the image sensing means determined by said determination means; 
5 selection means for selecting the image sensing means at a position, which corresponds to the position infor- 

mation registered together with the feature pattern of the object in said registration means, of said plurality of 
image sensing means upon verification of a feature pattern; 

second feature extraction means for extracting a feature pattern which represents a feature of the object from 
the object image obtained by the image sensing means selected by said selection means; and 
10 verification means for specifying an object associated with the object image by verifying the feature pattern 

extracted by said second feature extraction means with the feature pattern of the object registered in said 
registration means. 

15. An apparatus according to claim 1 4, characterized in that each of said first and second feature extraction means 
15 comprises: 

feature point detection means for detecting feature points of an object from the input object image; 

feature region setting means for setting a feature region on the basis of the feature points detected by said 

feature point detection means; 

20 region segmentation means for segmenting the feature region set by said feature region setting means into 

a plurality of regions; and 

feature pattern extraction means for computing brightness average values in the regions segmented by said 
region segmentation means, and extracting a feature pattern which represents a feature of the object on the 
basis of the brightness average values. 

25 

16. An image processing method characterized by comprising: 

the step (SO - S5) of sensing object images from different directions, extracting feature points from the sensed 
object images, and computing a feature pattern on the basis of the extracted feature points. 

30 17. A method according to claim 16, characterized by comprising: 

the first step (SO) of sensing object images from different directions; and 

the second step (S1 - S5) of extracting feature points from the object images sensed in the first step, setting 
a feature region on the basis of the extracted feature points, segmenting the set feature region into a plurality 
35 of regions, computing predetermined information in each segmented region, and computing a feature pattern 

on the basis of the computed predetermined information. 

18. A method according to claim 16, characterized by comprising: 

4 o the first step (SO) of sensing object images from different directions; 

the second step (S1 - S5) of extracting feature points from the object images sensed in the first step, setting 
a feature region on the basis of the extracted feature points, segmenting the set feature region into a plurality 
of regions, computing an average value of brightness levels in each segmented region, and computing a 
feature pattern on the basis of the computed average value; 

45 the third step (S7) of registering the feature pattern computed in the second step as a feature pattern associated 

with a predetermined object; and 

the fourth step (S11, S12) of specifying an object associated with the object image by comparing the feature 
pattern computed in the second step with the feature pattern registered in the third step. 

50 19. A method according to claim 18, characterized in that the first step includes the step of sensing object images 
from different directions which line up vertically, and 

the second step includes the step of computing the feature pattern using one of a feature point group including 
central points of right and left pupils and central points of right and left nasal cavities of the object image, and a 
feature point group including central points of right and left pupils of the object image. 

55 

20. A method according to claim 18, characterized in that the first step includes the step of sensing object images 
from different directions which line up horizontally, and 

the second step includes the step of computing the feature pattern using one of a feature point group including 



12 




EP1 134 691 A2 



central points of right and left pupils and central points of right and left nasal cavities of the object image, a feature 
point group including central points of right and left pupils and a central point of a left nasal cavity of the object 
image, and a feature point group including central points of right and left pupils and a central point of a right nasal 
cavity of the object image. 

5 

21. A method according to claim 18, characterized in that the first step includes the step of simultaneously sensing 
object images from directions different in vertical and horizontal directions, and 

the second step includes the step of computing the feature pattern using one of a feature point group including 
central points of right and left pupils and central points of right and left nasal cavities of the object image, a feature 
10 point group including central points of right and left pupils and a central point of a left nasal cavity of the object 

image, a feature point group including central points of right and left pupils and a central point of a right nasal 
cavity of the object image, and a feature point group including central points of right and left pupils of the object 
image. 

15 22. A method according to claim 18, characterized in that the second step includes the step of extracting feature 
vectors of different dimensions from respective object images sensed in the first step, and arranging the extracted 
feature vectors of different dimensions in turn to integrate them as a multi-dimensional feature pattern. 

23. A method according to claim 18, characterized in that the second step includes the step of capturing object 
20 images sensed in the first step at predetermined time intervals, computing feature pattern of the object images of 

identical times, and arranging feature patterns of different times in turn to integrate them as a time-series feature 
pattern. 

24. A method according to claim 16, characterized by comprising: 

25 

the first step (SO) of sensing an object image from different positions, and inputting a plurality of object images 
at different image sensing positions; 

the second step (S1 - S5) of extracting feature patterns that represent features of an object from the plurality 
of object images input in the first step; 
30 the third step (S1 1 ) of verifying the plurality of feature patterns extracted in the second step with a reference 

feature pattern which is registered in advance; and 

the fourth step (S12) of determining, when at least one of the plurality of feature patterns extracted in the 
second step matches the reference feature pattern which is registered in advance as a result of verification 
of the third step, that an object associated with that object image is a person himself or herself. 

35 

25. A method according to claim 24, characterized in that the first step includes the step of inputting a plurality of 
object images at different image sensing positions using a plurality of image sensing means which are set in 
advance at a plurality of predetermined positions, and sense an object image from a plurality of different positions. 

40 26. A method according to claim 24, characterized in that the second step includes: 

the step of detecting feature points of an object from the input object image; 
the step of setting a feature region on the basis of the detected feature points; 
the step of segmenting the set feature region into a plurality of regions; and 
45 the step of computing brightness average values in the segmented regions, and extracting a feature pattern 

which represents a feature of the object on the basis of the brightness average values. 

27. A method according to claim 16, characterized by comprising: 

50 the first step (SO) of sensing an object image from different positions, and inputting a plurality of object images 

at different image sensing positions; 

the second step (S41 ) of determining an image sensing position of an object image to be used from the plurality 
of object images input in the first step upon registration of a feature pattern; 

the third step (S1 - S5) of extracting a feature pattern which represents a feature of an object from the object 
55 image determined in the second step; 

the fourth step (S7 - S10) of registering the feature pattern extracted in the third step as a reference feature 
pattern associated with the object in correspondence with position information indicating the image sensing 
position of the corresponding object image; 
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the fifth step (S42) of selecting an object image at an image sensing position, which corresponds to the position 
information registered together with the feature pattern of the object to be verified registered in the fourth step, 
of the plurality of object images input in the first step upon verification of a feature pattern; 
the sixth step (S1 - S5) of extracting a feature pattern which represents a feature of the object from the object 
5 image selected in the fifth step; and 

the seventh step (S11, S12) of specifying an object associated with the object image by verifying the feature 
pattern extracted in the sixth step with the feature pattern of the object to be verified registered in the fourth step. 

28. A method according to claim 1 6, characterized in that an object image is input at different image sensing positions 
10 by moving at least one image sensing means to a plurality of predetermined positions and sensing an object image 

at respective positions. 

29. A method according to claim 16, characterized by comprising: 

* 5 the first step (SO) of inputting a plurality of object images at different image sensing positions using a plurality 

of image sensing means, respectively set in advance at a plurality of predetermined positions, for sensing an 
object image from a plurality of different positions; 

the second step (S41) of determining a position of the image sensing means to be used of said plurality of 
image sensing means upon registration of a feature pattern; 
20 the third step (S1 - S5) of extracting a feature pattern which represents a feature of an object from the object 

image obtained by the image sensing means determined in the second step; 

the fourth step (S7 - S10) of registering the feature pattern extracted in the third step as a reference feature 
pattern associated with the object in correspondence with position information indicating the position of the 
image sensing means determined in the second step; 
25 the fifth step (S43) of selecting the image sensing means at a position, which corresponds to the position 

information registered together with the feature pattern of the object in the fourth step, of said plurality of image 
sensing means upon verification of a feature pattern; 

the sixth step (S1 - S5) of extracting a feature pattern which represents a feature of the object from the object 
image obtained by the image sensing means selected in the fifth step; and 
30 the seventh step (S11 , S12) of specifying an object associated with the object image by verifying the feature 

pattern extracted in the sixth step with the feature pattern of the object registered in the fourth step. 

30. A method according to claim 29, characterized in that each of the third and sixth steps includes: 

35 the step of detecting feature points of an object from the input object image; 

the step of setting a feature region on the basis of the detected feature points; 
the step of segmenting the set feature region into a plurality of regions; and 

the step of computing brightness average values in the segmented regions, and extracting a feature pattern 
which represents a feature of the object on the basis of the brightness average values. 

40 
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